ORION software tool for the geometrical calibration of all-sky cameras

This paper presents the software application ORION (All-sky camera geOmetry calibRation from star positIONs). This software has been developed with the aim of providing geometrical calibration to all-sky cameras, i.e. assess which sky coordinates (zenith and azimuth angles) correspond to each camera pixel. It is useful to locate bodies over the celestial vault, like stars and planets, in the camera images. The user needs to feed ORION with a set of cloud-free sky images captured at night-time for obtaining the calibration matrices. ORION searches the position of various stars in the sky images. This search can be automatic or manual. The sky coordinates of the stars and the corresponding pixel positions in the camera images are used together to determine the calibration matrices. The calibration is based on three parameters: the pixel position of the sky zenith in the image; the shift angle of the azimuth viewed by the camera with respect to the real North; and the relationship between the sky zenith angle and the pixel radial distance regards to the sky zenith in the image. In addition, ORION includes other features to facilitate its use, such as the check of the accuracy of the calibration. An example of ORION application is shown, obtaining the calibration matrices for a set of images and studying the accuracy of the calibration to predict a star position. Accuracy is about 9.0 arcmin for the analyzed example using a camera with average resolution of 5.4 arcmin/pixel (about 1.7 pixels).

This statement is required for submission and will appear in the published article if the submission is accepted. Please make sure it is accurate.

Unfunded studies
Enter: The author(s) received no specific funding for this work.   looking to a mirror oriented to the sky or looking directly to the sky with a fish-eye lens; 6 static cameras or moving cameras, usually installed on a sun-tracker with a shadow ball 7 to block the direct sun light; operating at daytime, night-time or both; and others. 8 Some of the best features of the current all-sky cameras are: they allow the possibility 9 of changing exposure time, sensor gain and other parameters to adapt the camera to the 10 sky scenario: they are able to obtain a snapshot of the full sky radiance, covering every 11 sky angle and in various spectral ranges; the capture time is short; and they are 12 economic as compared to other instruments. Conversely, obtaining an accurate 13 October 28, 2021 1/13 radiometric calibration of the images is difficult, the filters are generally too wide for 14 some applications, and there are also issues with the presence of hot pixels, pixel 15 saturation, lens aberrations, and image vignetting, among others. 16 This kind of instruments is generally used to observe and quantify clouds and cloud 17 cover [2][3][4][5][6][7][8][9][10] or as a proxy of the sky conditions. However, all-sky cameras present high 18 versatility and they can also be used for different purposes: to derive other more 19 complex cloud properties as cloud base height by stereoscopic methods [11,12]; to detect 20 and observe aurora, celestial bodies or bolides [13][14][15]; to estimate the sky 21 radiance [16][17][18]; to study the cloud effects over solar radiation [19,20]; to study the 22 polarization of the sky light [21,22]; to detect and retrieve atmospheric aerosol 23 properties [23][24][25]; to obtain synergy in combination with other instruments like 24 radiometers, ceilometers or photometers [26][27][28], among others. 25 The knowledge of the sky coordinates that are represented in each pixel of an all-sky 26 camera is crucial in several applications; for example to extract sky radiance at specific 27 sky angles [25,28], to forecast solar irradiance [29,30], to calculate aurora and cloud 28 base altitudes by stereographic methods [31,32], or simply to locate any body over the 29 celestial vault. This can be achieved by a geometrical calibration of the all-sky cameras, 30 which basically consists in obtaining two matrices of the same size than the camera 31 images but with the values of the both azimuth and zenith angles viewed by each pixel. 32 For this purpose, several calibration methods have been described in the literature. 33 Intrinsic calibration, for determining the internal parameters of the camera, is usually 34 done recording images of a chessboard, such as OcamCalib toolbox [33,34], where the 35 corners of the chessboard are identified and used to detect any distortion on the camera 36 optics. Extrinsic camera calibrations, consisting of the determination of the camera 37 orientation in the local reference frame [30], are generally based on the identification of 38 the positions of the Sun [32,35,36] or any star [28,37,38] in several images and on the 39 correlation of these positions with the Sun or star coordinates. 40 In this framework, we have developed the software application named ORION 41 (all-sky camera geOmetry calibRation from star positIONs), with the main objective of 42 providing an open and free tool for the geometrical calibration of all-sky cameras that is 43 accurate, simple and user-friendly.The use of stars instead of the Sun for the geometric 44 calibration is chosen because the Sun size in camera images is usually larger (problems 45 to identify its center in the image) and, in addition, by using multiple stars we are able 46 to cover more sky angles. ORION is written in python3 language and Qt5, and it is  The all-sky camera model is an "OMEA 3C" from Alcor System manufacturer. It is 63 formed by a sensor with a fisheye lens, both encapsulated in a weatherproof case with a 64 BK7 glass dome on top. The case includes a heating system to avoid water condensation 65 on the dome. The camera sensor is a SONY IMX178 RGB CMOS sensor, with an 66 image size of 3096 X 2080 pixels, a pixel scale of 5.4 arcmin/pixel and 14-bit resolution. 67 This all-sky camera is configured to capture a multi-exposure set of raw sky images 68 every 2 minutes at night-time and every 5 minutes at daytime. These raw images are 69 stored and then properly converted to 8-bits true color or gray-scale images. The  The Manual mode allows the user to manually choose the ROI. for each sky image, 101 ORION opens an additional window showing the sky image where the user must select 102 the ROI as a rectangular pixel box. For this, ORION uses the function selectROI that 103 is native part of OpenCV library [43]. In order to know where the ROI must be 104 manually selected, or to check if ORION identifies the stars positions correctly, either in 105 Manual or Automatic mode, we recommend the use of Stellarium 106 (https://stellarium.org), which is a free open source planetarium showing a realistic 107 hemispherical sky similar to the one captured by a fisheye lens [44]. 108 Automatic mode skips the ROI selection image by image, simplifying the process. Default. Both options are based on the use of calibration matrices. ORION uses these 112 matrices to find the position of the pixel closest to the chosen star; Harvesine distance 113 function is used for this calculation [45]. If the accuracy of the previous calibration 114 matrices is admissible, the real image of the star in the sky image must be close to the 115 predicted pixel, although not necessarily in the same pixel. Hence, ORION looks for the 116 brightest pixel within a box centred in the pixel predicted by calibration matrices. The 117 size of the mentioned box depends on the size of the full image and also on the chosen 118 option (Custom or Default). ORION assumes that the center of the chosen star is 119 located in the image in its brightest pixel and it stores the x and y position of that pixel. 120 Custom option uses calibration matrices that were obtained in a previous calibration, 121 for example using Manual mode. In this case, the box used to find the star position is a 122 square box which side dimension is the height (or the width if it is higher) of the sky 123 image divided by 100. This arbitrary value was chosen after running various tests and 124 seeing empirically that it worked best for different images.  coordinates are the column and row numbers, respectively, and the system is centred in 151 the left-top pixel (x=0, y=0). The first element corresponds to zero position for x and y 152 because ORION is programmed in python3 language. It is convenient to convert this 153 system into polar coordinates, which is more similar to the observed in the sky by 154 zenith and azimuth angles. In this sense, a polar system centred in the zenith of the sky 155 with Cartesian coordinates equal to x=x C and y=y C , can be described as: 156 October 28, 2021 4/13 where r is the radial distance in pixels from the center, and Φ the polar angle in this 157 new system. This polar angle presents its zero value in the direction of y-axis in a 158 similar way than in Fig. 2a.

159
Radial distance and polar angle are directly related to the sky zenith and azimuth 160 angles, respectively. Assuming polar symmetry and that the camera is well levelled, the 161 polar angle must be equal to the sky azimuth but with a shift due to a non perfect 162 alignment of the camera with respect to the North. This shift is the same used in the 163 "Default" option of Automatic mode in Section 3.1. Sky zenith angle is related to radial 164 distance, but this relationship can be linear (which is assumed in the mentioned 165 "Default" option), or a higher polynomial degree. Hence, if we determined the x C and 166 y C coordinates, we will obtain the polar coordinates of Eq. 1 and 2; then, if we 167 determine the shift of polar angle with respect to the sky azimuth, and the relationship 168 between radial distance and zenith angle, we can transform the obtained polar 169 coordinates into the sky angles viewed by each pixel. This is the way that ORION uses 170 to calculate the calibration matrices.

171
The required x C and y C values, the azimuth shift from polar coordinates, and the 172 relationship between zenith angle and radial distance can be obtained from the stored 173 data in the previous Section 2.2: real azimuth and zenith star values and x and y pixel 174 positions where the stars were found.  An example of ORION calibration with real images is shown in this section. The images 232 corresponds to the all-sky camera, described in Section 2.1, installed at Valladolid. We 233 selected the images every 2 minutes from 20:20 UTC to 23:58 UTC at 23 rd August 2020. 234 We chose this set of images since it corresponds to a clear night with fully cloudless 235 conditions and without Moon presence. First of all, the camera location information 236 and the path for images local folder are introduced in the input parameters. After that 237 we can choose between manual or automatic mode (see Fig 1). Automatic default 238 option is selected since we have no previous custom calibration. The information about 239 default matrices in this example is: the shift from north equal to -5°and extreme zenith 240 equal to 95°. Both parameters are introduced in the application as can be observed in 241 the screenshot of  Once this previous information is introduced, we start to identify and detect star 243 positions in the sky images. The first chosen star in this example is Capella; this is 244 selected from the list of available stars and the path where the file with the data will be 245 stored is selected. The procedure described here is the same for every star. When the 246 October 28, 2021 6/13 "Start" button is pressed (see Fig 3), ORION reads, from "pyephem" library, the 247 azimuth and zenith angles corresponding to the time and place of the first image; then, 248 the position of the closest pixel to the star coordinates is obtained following the 249 calibration matrices given by the "Default" input parameters. The position of the 250 brightest pixel inside a square box centred in the mentioned "closest pixel" is considered 251 the position of the center of the chosen star in the image. The size of this square box is 252 the width (or height, whatever is larger) of the image divided by 50. ORION shows the 253 chosen star position marked with a white circle in the main sky image (left side of the 254 application; Fig 3) but also in a zoomed image at the bottom-right corner of the 255 application ("Region of image" in Fig 3). This zoomed image is useful to discern 256 whether the chosen star position is correct or not (hot pixels can lead to mistaken 257 identification). If the user considers that the star position is well identified, the user will 258 click on the "Add point" button and then the pixel position will be hold on in the main 259 sky image with a green circle; if not, the user will click on "Remove point" and the pixel 260 will be hold on with a red circle and the star position for this image will not be stored. 261 Another way to obtain the star position in the image is the "Manual" mode, as it is 262 shown in Fig 4. This method is similar to the automatic one but for each image an 263 additional window ("Select star" in Fig 4) is open showing the sky image to analyze.

264
Then the user manually chooses the box where the position of the brightest pixel is 265 found (see white box with cross inside in Fig 4).  In the present example, we repeat the process to obtain the positions of Capella, 277 Altair, Vega and Deneb. Once the four files are generated using the "Default" mode, 278 they are used to obtain the azimuth and zenith calibration matrices of the camera. For 279 this, the "calculate center and calibration matrices" tab in the application is selected.

280
In this tab we introduce the path of the saved files with star and camera pixel positions 281 ("Input files path" in Fig. 5); in this case the four mentioned files for each star (Capella, 282 Altair, Vega and Deneb). This tab also requests a path to save the calibration matrices 283 which will be calculated from the input files. After pressing the "Calculate" button, the 284 software calculates the center of the image and the relationship between the zenith 285 angle and radial distance, as explained in Section 2.3. The degree of the polynomial fit 286 between zenith angle and radial distance can be chosen, being equal to 2 in our example 287 (see right side of Fig. 5), since the degree 1 option (linear fit) does not perfectly agree 288 with the data under high zenith angle values (see Figure S01 in supplementary material). 289 The azimuth and zenith calibration matrices are also calculated from this information 290 and they are shown in the application, as can be observed in the left side of Fig. 5. The obtained calibration matrices in this example could be used; however, the graph 292 October 28, 2021 7/13 of Fig. 5 shows that they were calculated without taking into account zenith angles 293 between 40 and 70°. It is recommended to cover the maximum zenith angle range for 294 the calibration, hence, in this example, we will look for more star positions in order to 295 fill the gap of zenith angles. Two additional stars are selected for this purpose:

297
The pixel positions of Arcturus and Alphecca are retrieved using "Automatic" mode 298 but "Custom" option, since a previous calibration is available (Fig. 5), and this option 299 must be more accurate than the "Default". This mode is equal to "Default" but using a 300 pre-calculated azimuth and zenith matrices (the path is introduced as input, see Fig. 6) 301 and considering a smaller square box (100 times smaller than the maximum dimension 302 of the analyzed sky image, see Section 2.2) for finding the brightest pixel. Figure 6 303 presents an screenshot after the calculation of pixel and star positions for Alphecca is 304 finished. In this case, some pixels have not been properly identified by ORION (the star 305 was mixed up with a hot pixel), hence we have used the "Remove point" option for 306 them; they can be seen in red color in Fig. 6, indicating that these positions (red circles) 307 are not stored in the "alphecca.npy" file. In this example, the calibration matrices are recalculated adding the information of 309 these two new stars in the "Calculation center and calibration matrices" tab as shown 310 in Fig. 7. Now the fitting between zenith angles and radial distance is done with a 311 complete zenith angle range, and the second order polynomial relationship can be better 312 observed. As a final result, the center of the sky image, which corresponds to the sky 313 zenith, is really close to the center of the sky image: pixel (1000, 1000). Regarding the 314 shift from North, it is -5.23°, which is also close to our a priori approach of -5.0°when we 315 used "Default" option in "Automatic" mode.

317
Once the final calibration matrices are calculated, their performance for the detection of 318 stars in a sky image can be evaluated with the "Check calibration" menu. For this 319 purpose, one star must be chosen, for example Alioth which has not be used in the 320 calibration process, as shown in Fig. 8. ORION is run after the "Check" button is 321 pressed and it analyzes each single image from the set in the image path. The 322 calibration matrices point out that the center of the chosen star should be in a certain 323 pixel (marked in red circle in Fig. 8). ORION also looks for the brightest pixel in a box 324 centred in the mentioned pixel. This box has the size of the box used in the "Default" 325 option of "Automatic" mode (see 2.2). The brightest pixel (marked as green circle; see 326 Hence, the software calculates the real star position (brightest pixel) and the one 328 predicted by calibration matrices for all images, and then uses both positions to 329 quantify the agreement between the predicted star position by calibration and the real 330 one. Five panels with different analyses are provided for this verification, see Fig. 9. Performance of the obtained calibrations matrices for Alioth star positions: a) azimuth obtained with the calibration for the located star position (brightest point) as a function of the real star azimuth; b) zenith obtained with the calibration for the located star position (brightest point) as a function of the real star zenith; c) absolute pixel distance between the star position given by the calibration matrices and the position of the assumed real center of the star in the image (brightest point in a square box defined by the height or width of the sky image) as a function of star azimuth angle (red dotted line represents the mean absolute distance); d) absolute pixel distance between the star position given by the calibration matrices and the position of the assumed real center of the star in the image (brightest point in a square box defined by the height or width of the sky image) as a function of star zenith angle (red dotted line represents the mean absolute distance); e) absolute pixel distance between the star position given by the calibration matrices and the position of the assumed real center of the star in the image (brightest point in a square box defined by the height or width of the sky image) for each available used image (red dotted line represents the mean absolute distance).  This calibration check has been also carried out for 9 additional stars, see Table 1. It 346 is observed that the mean for all stars is between 4 and 12 arcmin. The best results are 347 obtained by Kochab with a mean of 4.50 arcmin and a standard deviation of 3.24 348 arcmin. The highest maximum values are observed in Dubhe, Mirach and Sirrah, which 349 corresponds to the presence of hot pixels in positions close to the stars that are not well 350 identified. In general, the mean accuracy for the 10 stars is about 9.0 arcmin (1.7 pixels) 351 and the mean precision, given by the standard deviation, is about 7.5 arcmin (1.4 352 pixels).  cameras, which will be helpful to locate any body (stars, planets, Sun, Moon...) in their 365 sky images if the sky coordinates of that body are known.